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Pair correlation densities of 
inhomogeneous quadratic forms 



By Jens Marklof 



Abstract 



Under explicit diophantine conditions on (a, (3) 6 I 2 , we prove that the 
local two-point correlations of the sequence given by the values (rre — a) 2 + 
(re — /3) 2 , with (m, n) G Z 2 , are those of a Poisson process. This partly confirms 
a conjecture of Berry and Tabor [2] on spectral statistics of quantized integrable 
systems, and also establishes a particular case of the quantitative version of the 
Oppenheim conjecture for inhomogeneous quadratic forms of signature (2,2). 
The proof uses theta sums and Ratner's classification of measures invariant 
under unipotent flows. 

1. Introduction 

1.1. Let us denote by < Ai < A2 < • • • — > 00 the infinite sequence given 
by the values of 

(m - a) 2 + (n - f3) 2 

at lattice points (m,n) £ Z 2 , for fixed a, (3 £ [0,1]. In a numerical experi- 
ment, Cheng and Lebowitz [3] found that, for generic a, (5, the local statistical 
measures of the deterministic sequence Xj appear to be those of independent 
random variables from a Poisson process. 

1.2. This numerical observation supports a conjecture of Berry and Tabor 
[2] in the context of quantum chaos, according to which the local eigenvalue 
statistics of generic quantized integrable systems are Poissonian. In the case 
discussed here, the Xj may be viewed (up to a factor 4-7T 2 ) as the eigenvalues 
of the Laplacian 

dx 2 dy 2 

with quasi-periodicity conditions 

ip(x + k,y + l) = e- 2n ^ ak+l3l ^(x, y), k,leZ. 



The corresponding classical dynamical system is the geodesic flow on the unit 
tangent bundle of the flat torus T 2 . 
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1.3. The asymptotic density of the sequence of Xj is it, according to the 
well known formula for the number of lattice points in a large, shifted circle: 

#{j : Xj <X} = #{(m, n) G I? : (m - a) 2 + (n - /3) 2 < A} ~ vrA 

for A — ► oo. The rate of convergence is discussed in detail by Kendall [11]. 

1.4. More generally, suppose we have a sequence Ai < A2 < • • • — > 00 of 
mean density D, i.e., 

lim : Xj <X} = D. 

For a given interval [a, b] C R, the pair correlation function is then defined as 
R 2 [a, 6] (A) = 7^ k : Xj < X, Xk < X, a< Xj - X k < b}. 

The following result is classical. 

1.5. Theorem. // i/te Xj come from a Poisson process with mean den- 
sity D, 



lim R 2 [a,b](X) = D(b - a) 

A— »oo 



almost surely. 



1.6. We will assume throughout most of the paper that a, (3, 1 are linearly 
independent over Q. This makes sure that there are no systematic degeneracies 
in the sequence, which would contradict the independence we wish to estab- 
lish. The symmetries leading to those degeneracies can, however, be removed 
without much difficulty. This will be illustrated in Appendix A. 

1.7. We shall need a mild diophantine condition on a. An irrational 
number a G R is called diophantine if there exist constants k, C > such that 



P 

a 

Q 



C 
> — 

q K 



for all p, q G Z. The smallest possible value of k is n = 2 [26]. We will say a 
is of type n. 

1.8. Theorem. Suppose a, (3,1 are linearly independent over Q, and 
assume a is diophantine. Then 

lim i?2[o, 6] (A) = 7r(6 — a). 

A— >oo 

This proves the Berry- Tabor conjecture for the spectral two-point corre- 
lations of the Laplacian in 1.2. 

It is well known that almost all a (in the measure-theoretic sense) are 
diophantine [26]. We therefore have the following corollary. 
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1.9. Corollary. Let a, (3 be independent uniformly distributed random 
variables in [0, 1] . Then 



lim R 2 [a,b]{\) = vr(6 

A— »oo 



almost surely. 



1.10. Remark. In [4], Cheng, Lebowitz and Major proved convergence of 
the expectation value 1 

lim ER 2 [a, 6] (A) =ir{b-a), 

A^oo 

that is, on average over a, (3. 

1.11. Remark. Notice that Theorem 1.8 is much stronger than the corol- 
lary. It provides explicit examples of "random" deterministic sequences that 
satisfy the pair correlation conjecture. An admissible choice is for instance 
a = a/2, p = [26]. 

1.12. The statement of Theorem 1.8 does not hold for any rational a, (3, 
where the pair correlation function is unbounded (see Appendix A. 10 for de- 
tails). This can be used to show that for generic (a, f3) (in the topological 
sense) the pair correlation function does not converge to a uniform density: 

1.13. Theorem. For any a > 0, there exists a set C C T 2 of second 
Baire category, for which the following holds. 2 

(i) For (a, (3) G C, there exist arbitrarily large A such that 

R 2 [-a,a}(\)> ' 0gA 



log log log A 

(ii) For (a, (3) £ C, there exists an infinite sequence L\ < L 2 < • • • — > oo 
such that 

lim R 2 [—a,a](Lj) = 2na. 

j— >oo 

In the above, log log log A may be replaced by any slowly increasing posi- 
tive function ^(A) < logloglogA with ^(A) — ► oo (A — ► oo). 

1.14. The above results can be extended to the pair correlation densities 
of forms (mi — ai) 2 + . . . + (m& — c^) 2 in more than two variables; see [16] for 
details. 



1 They consider a slightly different statistic, the number of lattice points in a random circular 
strip of fixed area. The variance of this distribution is very closely related to our pair correlation 
function. 

2 A set of first Baire category is a countable union of nowhere dense sets. Sets of second category 
are all those sets which are not of first category. 
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1.15. A brief review. After its formulation in 1977, Sarnak [25] was the 
first to prove the Berry- Tabor conjecture for the pair correlation of almost all 
positive definite binary quadratic forms 

am 2 + (3mn + jn 2 , m,n G Z 

( "almost all" in the measure-theoretic sense) . These values represent the eigen- 
values of the Laplacian on a flat torus. His proof uses averaging techniques to 
reduce the pair correlation problem to estimating the number of solutions of 
systems of diophantine equations. The almost-every where result then follows 
from a variant of the Borel-Cantelli argument. For further related examples 
of sequences whose pair correlation function converges to the uniform density 
almost everywhere in parameter space, see [20], [22], [30], [31], [34]. Results 
on higher correlations have been obtained recently in [21], [23], [32]. 

Eskin, Margulis and Mozes [8] have recently given explicit diophantine 
conditions under which the pair correlation function of the above binary 
quadratic forms is Poisson. Their approach uses ergodic-theoretic methods 
based on Ratner's classification of measures invariant under unipotent flows. 
This will also be the key ingredient in our proof for the inhomogeneous set-up. 
New in the approach presented here is the application of theta sums [13], [14], 
[15]- 

The pair correlation problem for binary quadratic forms may be viewed 
as a special case of the quantitative version of the Oppenheim conjecture for 
forms of signature (2,2), which is particularly difficult [7]. 

Acknowledgments. I thank A. Eskin, F. Gotze, G. Margulis, S. Mozes, 
Z. Rudnick and N. Shah for very helpful discussions and correspondence. Part 
of this research was carried out during visits at the Universities of Bielefeld 
and Tel Aviv, with financial support from SFB 343 "Diskrete Strukturen in der 
Mathematik" and the Hermann Minkowski Center for Geometry, respectively. 
I have also highly appreciated the referees' and A. Strombergsson's comments 
and suggestions on the first version of this paper. 

2. The plan 

2.1. The plan is first to smooth the pair correlation function, i.e., to 
consider 

R 2 ^ u ij 2 ,h,x) = (x) ^ (y) ~ A *)- 

Here ipi,i^2 £ <S(R+) are real- valued, and <S(R+) denotes the Schwartz class 
of infinitely differentiable functions of the half line R+ (including the origin), 
which, as well as their derivatives, decrease rapidly at +oo. It is helpful to 
think of ^1,^2 as smoothed characteristic functions, i.e., positive and with 
compact support. Note that h is the Fourier transform of a compactly sup- 
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ported function h £ C(M), denned by 



Us) = I 

JR 



h(u)e(^us) du, 



with the shorthand e(z) := e 2nlz . 

We will prove the following (Section 8). 

2.2. Theorem. Let ipi,ip2 & <5(R+) be real-valued, and h G C(R) with 
compact support. Suppose a, (3, 1 are linearly independent over Q, and assume 
a is diophantine. Then 



The first term comes straight from the terms j = k; the second one is the 
more interesting. 

Theorem 2.2 implies Theorem 1.8 by a standard approximation argument 
(Section 8). 

2.3. Using the Fourier transform we may write 



We will show that the inner sums can be viewed as a theta sum (see 4.14 for 
details) 



living on a certain manifold £ of finite volume (Sections 3 and 4). The inte- 
gration in 



will then be identified with an orbit of a unipotent flow on S, which becomes 
equidistributed as A — > oo. The equidistribution follows from Ratner's classi- 
fication of measures invariant under the unipotent flow (Section 5). A crucial 
subtlety is that £ is noncompact, and that the theta sum is unbounded on 
this noncompact space. This requires careful estimates which guarantee that 
no positive mass of the above integral over a small arc of the orbit escapes to 
infinity (Section 6). 

The only exception is a small neighbourhood of u = 0, where in fact a 
positive mass escapes to infinity, giving a contribution 




r 2 (^2, h, a) = -L jf (]r ^ (ky^)) (]r ^(hy^x jU j)h{u) du 






which is the second term in Theorem 2.2. 
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The remaining part of the orbit becomes equidistributed under the above 
diophantine conditions, which yields 



^7 / O^e^dfi [ h(u)du, 

2->) JY, JR 



where \i is the invariant measure (Section 7). The first integral can be calcu- 
lated quite easily (Section 8). It is 

/Ttt - / O^Ofodn [ h(u)du = ir [ Y>i(r)y>2(7~) dr [ h(u)du, 
(j,{2j) Jt, Jr Jo Jr. 

which finally yields 

P oo 

wh(0) / ipi(r)ip2(r) dr, 
Jo 

the first term in Theorem 2.2. 

The proof of Theorem 1.13, which provides a set of counterexamples to 
the convergence to uniform density, is given in Section 9. 



3. Schrodinger and Shale-Weil representation 

3.1. Let u> be the standard symplectic form on R 2fc , i.e., 

where 



x 

y 



*=l " I. £' = */ > x,y,x',y'GR fc . 



is then defined as the set R x R with mul- 



The Heisenberg group 
tiplication law [12] 

= « + + J 7 + 

Note that we have the decomposition 

V N 



x 



x 




,0 



,0 (0,t-ix-y) 



3.2. The Schrodinger representation of H(R fe ) on / G L 2 (R fe ) is given by 
(cf. [12, p. 15]) 



W 



W 



x 




y 



,ojf 
,o]/ 



(w) =e(x-w)/(w), with x, w£l l , 



w) =/(w-y), 
W(0,t) = e(t)id, 



with y, w G 
with t G R. 
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W 



x 

y 



,t)f 



(w) = e(t - ix ■ y) e(x • w) /(w - y). 



3.3. For every element M in the symplectic group Sp(fc,M) of R , we can 
define a new representation Wm of H(lR fc ) by 

W M (S,t)=W(M£,t). 

All such representations are irreducible and, by the Stone- von Neumann theo- 
rem, unitarily equivalent (see [12] for details). That is, for each M G Sp(/c,M) 
there exists a unitary operator R(M) such that 

R(M) W($,t) R{M)- 1 = W(M£,t). 

The R(M) is determined up to a unitary phase factor and defines the projective 
Shale-Weil representation of the symplectic group. Projective means that 

R(MM') = c(M,M')R(M)R(M') 

with cocycle c(M,M') G C, \c(M,M')\ = 1, but c(M, M') ^ 1 in general. 

3.4. For our present purpose it suffices to consider the group SL(2,R) 
which is embedded in Sp(/c,M) by 



where 1^ is the k x k unit matrix. 

The action of M G SL(2, M) on £ G 



o l fe 6 l fc 
c lfc d lfc 



ax + by 
cx + dy 



with M 



is then given by 



X 

y 



3.5. For M G SL(2,1 
(see [12, p. 61f]). 

[R(M)f](w) 



Sp(k,M) we have the explicit representations 



' |a| fc/2 e(i||w|| 2 a6)/(aw) 



-k/2 



(a||w|| 2 + d||w'|| 2 ) — w • w' 



(c = 0) 
/(w')dw' (c/0). 



Here 



denotes the euclidean norm in 
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3.6. If 

*)• A H- *)• *H 2 - R) 

with M\M.2 = Ms, the corresponding cocycle is 

c(M 1 ,M 2 ) =e~ i ^ si s n ( ClC2C3 )/ 4 , 

where 

f -1 (x<0) 
sign(x) = < (x = 0) 
( 1 (x>0). 



3.7. In the special case when 

M = / cos 0i -sin 0i \ m = ( cos< ^ 2 _sin ^2 

1 I sin 4>\ cos 4>\ J 2 I sin 02 cos 02 

we find 

c(Mi,M 2 ) = e _ " rfc ( <7 *i +<T *2 _(T *iW/ 4 

where 

2^ if = vir, 



2u + l if utt < < (f + 1)7T . 



3.8. Every M G SL(2,M) admits the unique Iwasawa decomposition 

/ 1 u \ I v 1 / 2 \ / cos0 — sin0 \ . 
M= (o 1 )[ t,-V=» J(sin0 cos0 rj=(^)' 

where r = ti + ivGi^,0G [0, 27r). This parametrization leads to the well 
known action of SL(2,M) on S) x [0, 27r), 

" 5 ] (t, 0) = ( aT 7 , + arg(cr + d) mod 2tt). 
\ c a J ct + a 

We will sometimes use the convenient notation (Mr, 0m) := M(r,(f>) and 
ti M := Re(Mr), t> M := Im(Mr). 

3.9. The (projective) Shale- Weil representation of SL(2,R) reads in these 
coordinates 

[i?(r,0)/](w) = [ J R(r,O) J R(i,0)/](w) =^/ 4 e(I||w|| 2 n)[ J R(i,0)/]( W 1 / 2 w) 
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and 

[#M)/](w) 

' /(w) 
/(-w) 

I sin ( 



-fc/2 



A(||w|| 2 + ||w'|| 2 )cos0 — w • w' 



sin0 



(0 = mod 2vr) 
(0 = 7T mod 27r) 

/(w')dw' 
(0/0 mod 7r). 



Note that i?(i, 7r/2) = JF is the Fourier transform. 

3.10. For Schwartz functions / G S(R fe ), 

5 (||w|| 2 + ||w'|| 2 ) COS — w • w' 



lim I sin 01 fc//2 / < 
</>->o± 



am 



and hence this projective representation is in general discontinuous at = vir, 
v G Z. This can be overcome by setting 

£( T ,0) = e - i7rfe<7 ^ 4 i?(r,0). 

In fact, ^ corresponds to a unitary representation of the double cover of 
SL(2,R) [12]. This means in particular that (compare 3.7) 

i?(i,0)i?(i,0') = i?(i,0 + / ), 
where G [0, 4tt) parametrizes the double cover of SO(2) C SL(2,R). 

4. Theta sums 

4.1. The Jacobi group is defined as the semidirect product [1] 

Sp(£;,R) x H(R fe ) 

with multiplication law 

(M; & i)(M'; t') = {MM'- £ + Mf, t + t'+ ±u,(£, Mi')). 
This definition is motivated by the fact that, since 

R(M)W{£',t') = W{M£',t')R{M), 

(recall 3.3) we have 

W(£,t)R(M) W(£',t')R(M') 

= W(£,t)W(M$',t') R{M)R{M') 

= c(M, M')- 1 W{£ + Md'. t + t' + M£')) R(MM'). 
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Hence 

R(M;£,t) = W(&t)R(M) 

defines a projective representation of the Jacobi group, with cocycle c(M, M 1 ) 
as above, the so-called Schrddinger-Weil representation [1]. 
Let us also put 

R(T,<f>;Z,t) = W(tt)R(r, <t>). 



4.2. JacobVs theta sum. We define Jacobi's theta sum for / G S(R k ) by 
e f (T,fr£,t)= [R(r,4>;tt)f](m). 



More explicitly, for r = u + iv , £ = ^ X J , 

e / (r,^;^*)=t; fc / 4 e(t-ix-y) £ /*((m - y)v l ' 2 )e{ \ ||m - y || 2 u + m • x), 

mei k 

where 

f 4) = R(i,<l>)f. 



It is easily seen that if / G «S(M ) then G <S(M ) for fixed, and thus also 
R(r,(f);£,t)f G for fixed (r, <j>\ This guarantees rapid convergence 

of the above series. We have the following uniform bound. 

4.3. Lemma. Let = R(i, with f G S(R k ). Then, for any R > 1, 
there is a constant cr such that for all w G M. k , <fi G M, 

iMw)i< Ci? (i + iiw|ir R . 

Proof. Since / G S(IR fc ), we can use repeated integration by parts to show 



that 



sm< 



-k/2 



I 

Jr> 



l(ll w l| 2 + ll w 'H 2 ) COS0 - w • w' 



smi 



/(w')dw' 



< 4(i+llw| 



-R 



uniformly for all 4> $ iyn — y^j, vir + ^L), v G Z. That is, 

|/^(w)|<4(l + ||w||)- R 

in the above range. 

Furthermore f n /2 is up to a phase factor e mh the Fourier transform of / 
and therefore of Schwartz class as well. Again, after integration by parts, 



sin <f>\ 



-k/2 



t?(||w|| 2 + ||w'|| 2 ) COS0 - w • w' 



sin< 



L/2(™')dw' 



<c R (i + \MT R 
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for all 4> 4- ( U7T ~ too > U7T TSo)' v ^ ^ - This means 

IW2(W)|<4(1 + ||W||)- R 

in the above range, or, by replacement of i— ► — 7r/2, 

|/ (w)|<4(l + ||w||)- R , 

for all ^ (z^7r + \t; — j^, vir + \n + !/£Z. 

Clearly for each £ K at least one of the bounds applies; we put cr = 
max{c' fi ,c^}. □ 

4.4. The following transformation formulas are crucial for our further 
investigations: 

Jacobi 1. 

©/ (-p^ + argr; ( ~ y ) , tj = e~^% (V, 0; 

Proof. The Poisson summation formula states that for any f £ S 
E P7](m) = E /( m ) 

where J 7 is the Fourier transform. Because 

F = R(i,n/2) = R{S), S= ^ ) , 

and secondly R(r, 4>; t)f G <S(M fe ) for fixed (r, 0; £, t), the Poisson summation 
formula yields 

E [i?(S)i?(T,0;£,*)/](m)= E fe0;^,*)/](m). 

We have 

R{S)R(t, <j>; £, t) = t)i?(r, 0)i?(i, 0) = t)R(S)R{r, 0)R(i, 0); 

furthermore 

R{S)R(t,0) = R ^-^,argr^ = i? (~^ ) fl(i,argr), 

since (r, 0) and (— -,0) are upper triangular matrices, and hence the cor- 
responding cocycles are trivial, i.e., equal to 1 (recall 3.6). Finally, since 
< arg r < it for r € f), 

fl(i,argr)£(i,0) = e i,rfc/4 .R(i, argr)«(i, 0) = e i7rfc/4 «(i, + argr). 

Collecting all terms, we find 

R(S)R(t, <f>; £, t) = ^ 4 R (-i + argr; S£, t) , 
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and hence 



£ [ J R(-^ + a rgr;^,t)/|(m)=e- i ^ 4 ]T [£(r, <fc £,*)/] (m), 



which proves the claim. 
Jacobi 2. 



with 



Proof. Clearly for any / G S 



E 



R i + 1,0; Q 1,01/ 



(m) = /( m )' 



and hence also (replace / with R(T,(ft;£,t)f) 



E 



R i + 1,0; Q ,0\R(r,<l>;t,t)f 



□ 



,i+±s-y) =6/ (r,^ * ) 



(m)= J] [%,0;£,t)/](m). 



We conclude by noticing 



+ ^ j ,oji?(r,</>; * j ,tj 



X 



where we have used that c((i, 0), (r, 0)) = 1 since (i, 0) is an upper triangular 
matrix; cf. 3.6. □ 

Jacobi 3. 

0/ ( T ' 0; ( i ) + ^' r + t + ^ (( i ) = (-i) 1 " 1 ©/^;^) 

/or any k, 1 6 Z fc , r£l 

Proof. By virtue of 3.2 we have for all / 
k 



E 



i 



,r / 



(m) = e (-|k-l) J2 /M. 



INHOMOGENEOUS QUADRATIC FORMS 

and therefore, replacing / with W(£, t)R(r, (j))f, 
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E 



w 



,r)W(tt)R(T, 4>)f 



(m) 



e(-ik-l) Y, [W(tt)R(r^)f]( m \ 



which gives the desired result. 



□ 



4.5. In what follows, we shall only need to consider products of theta sums 
of the form 



6/(^0; S,t)Q g (r,<i>;S,t), 

where f,g € S(R k ). Clearly such combinations do not depend on the i-variable. 
Let us therefore define the semi-direct product group 

G k = SL(2,R) k R 2k 

with multiplication law 

(M;e)(M';Z') = (MM';S + Mt'), 

and put 



Y U(( m ~ y)v 1/2 )e{±\\m - y\\ 2 u + m • x). 



By virtue of Lemma 4.3 and the Iwasawa parametrization 3.8, is a 

continuous C- valued function on G k . 

4.6. A short calculation yields that the set 

b d ) ■■(!)+ »>)■■(: $ ) * *»} c 

with s = i, . . . , |) G ]R fc , is closed under multiplication and inversion, and 
therefore forms a subgroup of G k . Note also that the subgroup 

N = {1} k l? k 

is normal in T k . 

4.7. Lemma. T k is generated by the elements 



-1 

1 



;0 , 



l l 
o 1 



s 
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Proof. The map 



a b \ I I a b \ ( abs \ ^ 2 k 



defines a group isomorphism. The matrices ( ° ~ x ) and ( J J ) generate 
SL(2,Z), hence the lemma. □ 

4.8. Proposition. The left action of the group T k on G k is properly 
discontinuous. A fundamental domain of F k in G k is given by 

Ft* =^sl(2,z) x {4> £ [0, 7r)} x [-i,i) 2fc }. 

where ^ 7 sl(2,z) *s ^ e fundamental domain in F) of the modular group SL(2, Z), 
gii>en by {t £ ft : u £ [— |, ^), |r| > 1}. 

Proof. As mentioned before, the matrices ( ° "J ) and ( J J ) generate 

SL(2,Z), which explains J 7 sl(2,z)- Note furthermore that ( ~* ^ ) generates 
the shift i — > cp -\- 7r. □ 



4.9. Proposition. For f,g e S(M. k ), Qf(T,4>;£)@ g (T,(j);£) is invariant 
under the left action of T k . 

Proof. This follows directly from Jacobi 1-3, since the left action of the 
generators from 4.7 is 

( T '* ; (y))"(4* + aIgT; ("/))■ 

<r.*e~( T+ i,*(;) + (j ;)(;)), 

and 

(t, </>;£) ^ (t, <A;£ + m), 
respectively. □ 
We find the following uniform estimate. 
4.10 Proposition. Let f,g e For any i? > 1, 



(-), [ r.o:( ] ; ' r.o: * 



v fe/2 V /iffm - y)v^ 2 )aJ(va - y V/2) + Or{v~ r ) 



m 
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uniformly for all (r, <fr; £) G G fc rai/i t> > \ . In addition, 



= o" 2 /*((n - y)o 1/2 ) 94 ,((n - y)«'/2) + O^tT 8 ), 
uniformly for all (r, 0; £) G G k with v > ^, y G n + [— ^, ^] fc and n G 

Proof. Suppose y G n + [— |, for an arbitrary integer n G Z fc . 
By virtue of Lemma 4.3 we have for any T > 1 

|/*((m - y y/ 2 )| < c T (l + ||m - y|b 1/2 )- T = Or(||m - n||- T v~ T l 2 ), 

which holds uniformly for v > \, <fr G M. and y G n + [— i, if m / n. 
Likewise for y^, 

|^((m - y)^/ 2 )| < c T (l + ||m - y||t; 1/2 )- T = T (||m - n||" T v~ T l 2 ), 

again uniformly for v > \, 4> G M. and y G n + [— \, ^] k , if rh / n. 

Hence the leading order contributions come from terms with rh = m, the 
sum of all other terms contributes Ot(v~ t ^ 2 ). □ 

The following lemmas will be useful later on. 

4.11 Lemma. The subgroup 

T e k Z 2fc , 

where 

r e = |^" ^ G SL(2, Z) : ab = cd=0 mod 2| 

denotes the theta group, is of index three in T k . 

Proof. It is well known [9] that Te is of index three in SL(2, Z) and 

SL(2,z)=[Jr/5 ) Z 1 ) 3 . 

By virtue of the group isomorphism employed in the proof of Lemma 4.7, we 
infer that 

4.12 Lemma. T fc is o/ /irate mdez m SL(2,Z) x (±Z) 2fe . 

Proof. The subgroup T e k Z 2k C T fc is of finite index in SL(2, Z) x Z 2k and 
thus also in SL(2,Z) k (±Z) 2fc . □ 
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4.13. Remark. Note that 

SL(2,Z)K(IZ) 2fc =((j ° ^;0^(SL(2,Z)KZ 2fe )^ ^ . ) , 

i.e., SL(2,Z) k (±Z) 2fe is isomorphic to SL(2,Z) k Z 2fe . 

4.14. In this paper, we will be interested in the case of quadratic forms in 
two variables, i.e., k = 2. The corresponding theta sum (defined for general k 
in 4.5) reads then 

e f (T,<t>;£) = v 1 ' 2 Yl U((™-Vi> 1/2 >(n-V2)v 1/2 ) 

x e{\(m - y\) 2 u + \(n — y 2 ) 2 u + mx\ + nx 2 ), 

where £ = \x\ ,X2,yi, y 2 ) G M 4 . This theta sum is related to the one introduced 
in Section 2 by 



Ofa(u, \)0fa(u, \) = G/(t,</>;£)G s (t, </>;£) 

with 

r = n + ii, cf> = 0, Z=X0,0,<x,P), 

and 

f(w 1 ,w 2 ) = ipi(wf + tof), g(,w 1 ,w 2 ) = ip 2 {w\ + u> 2 ). 

Recall that /<^ = o = / and likewise 94^^=0 = 9. 

The crucial advantage in dealing with Qj rather than the original 9^ is 
that the extra set of variables allows us to realize G f as a function on a finite- 
volume manifold and to employ ergodic-theoretic techniques. 

5. Unipotent flows 

5.1. Put 

«-((ilR 

For i e M, I J generates a unipotent one-parameter-subgroup of G k , denoted 
by J. For any lattice T in G fc , we now define the flow : T\G fc -► T\G fc by 
right translation by \I>q, 

Hence for 5 = (M; £) we have 

*<( 9 ) = (M ( J J ) ;«) . 

When projected onto r\SL(2,R), this flow becomes the classical horocycle 
flow. 
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5.2. Similarly, 



e-" 2 



generates a one-parameter-subgroup of G k . The flow : T\G k — > 
defined by 

represents a lift of the classical geodesic flow on T\ SL(2,R). 
5.3. We are interested in averages of the form 

F(u + iv,0;£) h{u)du 



/ 



where F is a continuous function T\G k — > R, and h is a continuous probability 
density with compact support. Setting g = (i, 0;£), and v = e _t , we may 
write the above integral as 

Pt (F) = J F(go*%& ) h(u) du = J F o & o * u ( 5o ) /i( u ) du, 

which may therefore be interpreted as the average along an orbit of the unipo- 
tent flow ty u , which is translated by <&*. Since Pt(l) = 1, Pt defines a probability 
measure on F\G k . 

5.4. Proposition. Xe£ r be a subgroup o/SL(2,Z) k Z 2fc of finite index. 
Then the family of probability measures {p t : t > 0} is relatively compact, i.e., 
every sequence of measures contains a subsequence which converges weakly to 
a probability measure on T\G k . 

Proof. Consider the function 

Xr{t) = ]T Xtf(Im( 7 T)), 

7e{r 0O u(-i)r 0O }\SL(2,z) 

where \R ls the characteristic function of the open interval (R, oo), and 
r °° = {( T ) :mG 4 CSL(2,Z). 



For -u + fy G .Fsl(2,z)> we thus have 

X R (u + iu) 



1 (v>R) 
[v< R). 



Because T is a finite index subgroup of SL(2,Z) k Z 2fc , X fl represents the 
characteristic function of a set in T\G k , whose complement is compact. 
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By construction, the function X R is independent of <p and £; we can there- 
fore apply the equidistribution theorem for arcs of long closed horocycles on 
r\f) (see, e.g., [10] and [15, Cor. 5.2]), which yields for g = (i>0;£), 



lim pt{X R ) = lim ( X R (u+\v) h(u) du = 1 / X R (u+iv 

V -+°J M^SLfrZ)) ^ sl(2 ,Z) 



du dv 



t 

Now 



dudv 

X R [u + w) — — 



SL(2,Z) 



dv „ 
= R~ 



Hence, given any e > 0, we find some i? > 1 such that 

supp t (X R ) < e. 

The family of pt is therefore tight, and the proposition follows from the Helly- 
Prokhorov theorem [28]. □ 

5.5. Proposition. If v is a weak limit of a subsequence of the probability 
measures pt with t — > oo, then v is invariant under the action of ^S? R , i.e., 
v o vJ/ R = v . 

Proof. Suppose {/? ti : i G N} is a convergent subsequence with weak 
limit v. That is, for any bounded continuous function F, we have 



lim p ti (F) = v(F). 



For any fixed s G R, we find 



Pt(Fo^ 



F(g n%n) Ku) du = / F(g * a 



- u+sexp(-t) w 



$o) h(u) du 



= J F(c/ *o$o) h ( u - s exp(-t)) du. 

Furthermore 

p t (Fo^)- Pt (F) = 



F(g nn) h(u-sexp(-t))-h(u) 



du 



du. 



< (sup|F|) J h(u — s exp(— t)) — h(u) 

Hence, given any e > 0, we find a T such that 

\p t (Fo^)- Pt (F)\<e 

for all t > T. Because the function F = F o ^> s (s is fixed) is bounded 
continuous, the limit 



lim fti (Fo$ s ) = v{Fo 
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exists, and we know from the above inequality that 

\u(Fo f s ) - v(F)\ < e 
for any e > 0. Therefore v(F o = v(F). □ 

5.6. Ratner [18], [19] gives a classification of all ergodic ^-invariant mea- 
sures on T\G k . We will now investigate which of these measures are possible 
limits of the sequence {pt}- The answer will be unique, translates of orbits of 
\I> R become equidistributed. 

5.7. Theorem. Let T be a subgroup o/SL(2,Z) k l? k of finite index. 
Fix some point 

9o =(i,o ; |°Jjer\G fe 

such that the components of the vector (V, 1) £ are linearly independent 

over Q. Zei h be a continuous probability density R — > R+ iratt compact 
support. Then, for any bounded continuous function F on T\G k , 

lira [ Fo$'o tf"( 5o ) /*(«) d U = 1 / F d/i 
^ooJ K /i(r\G' c ) Jr\G k 

where jjl is the Haar measure of G k . 

This theorem is a special case of Shah's more general Theorem 1.4 in 
[27] on the equidistribution of translates of unipotent orbits. Because of the 
simple structure of the Lie groups studied here, the proof of Theorem 5.7 is 
less involved than in the general context. 

5.8. Before we begin with the proof of Theorem 5.7, we consider the 
special test function 

7 GSL(2,Z) 

with (in the Iwasawa parametrization 3.8) 

fs{M) = fs{r, 4>) = xi{u + vcoi<j>) X2(v~ 1/2 cos</>) Xz{v~ l/2 sin</>) 

where Xj (J = 1> 2, 3) is the characteristic function of the interval [sj, Sj + Sj]. 
We assume in the following that Sj ranges over the fixed compact interval Ij, 
and that 1$ is furthermore properly contained in R + , i.e., S3 > s for some 
constant s > 0. Clearly f$ has compact support in SL(2,R). The function 
r] D : T 2k — ► R is the characteristic function of a domain D in T 2k with smooth 
boundary. 

Clearly, F$ may be viewed as a function on r\G fe , for T is a subgroup of 
SL(2,Z) k Z 2fc . 
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5.9. Lemma. Suppose the components of the vector (V, 1) £ R k+1 are 
linearly independent over Q. Then, given intervals Ii,l2,l3 as above, there 
exists a constant C > such that, for any domain D C T 2k with smooth 
boundary, 5\ , 62 , S3 > (sufficiently small) and si G I\, S2 G I2, S3 € I3, 

limsup / Fa|u + iu,0;( ° )) h{u) du < C '6162(83 + S3) [ r} D (£)d£. 
v^o Jul \ \ y / / Jj 2k 

The constant C may depend on the choice of h, y, I\, I2, 13. 

5.10. Proof. 

5.10.1. Given any e > and any domain D C T 2k with smooth boundary, 
we can cover D by a large but finite number of nonoverlapping cubes Cj C T 2k , 
in such a way that 



We may therefore assume without loss of generality that 1]d(£) is the charac- 
teristic function of an arbitrary cube in T 2k , i.e., //d(£) = r /i( x ) ? ?2(y) ) where 
771, 772 are characteristic functions of arbitrary cubes in T h . 

5.10.2. We recall that for 7 = ("J), 

In particular (with = 0), 

i^ 1 / 2 cos^ 7 = v~^ 2 (cu + d), v~ 1//2 sin (A.. = n 1 / 2 , 



a(« + w) + 6 a 1 cu + d a 

U 7 = Re — r : = 1 rpr = Uv cot 0~. 

c(« + iw) + a c c |cr + a| z c 
One then finds that 



F s (u + iv,0;^° jj=J2xi(^)X2(v' 1/2 (cu + d)) X 3(cv 1 / 2 ) m (by) m (dy), 

which, after being integrated against h(u)du, yields 

X3(c« 1/2 ) r7i (6y) r/ 2 (dy) / x 2 (c« 1/2 t) h(vt - -) dt. 

The compactness of the support of h implies that | = + 0(1), and hence 
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\d\ *C \s2 + 52\v 1 / 2 + |c|, i.e., \d\ <C |c| for v small. Therefore 



/ 



F 5 iu + iv,0; I ^ j j h(u)du 



^5 2 v 1/2 Yl Xi (j) X3(cv 1/2 ) 7/1 (by) r/ 2 (dy), 



|d|<A|c| 



where A > and the implied constant depend only on /i, if v is small enough. 

5.10.3. There are only finitely many terms with d = 0, which thus give 
a total contribution of order v 1 ^ 2 ; we will thus assume in the following d ^ 0. 
Likewise, if b = 0, we have ad = 1 and c € Z. This leads to a contribution of 
order v l l 2 \ logv |, which tends to zero in the limit v —> 0. 

The solutions of the equation ad — be = 1 with b,d ^0 can be obtained in 
the following way. Take nonzero coprime integers b,d £ Z, gcd(6, d) = 1, and 
suppose ao,co solves aod — 6co = 1. (Such a solution can always be found.) 
All other solutions must then be of the form a = ao + mb, c = cq + md with 
m € Z. We may assume without loss of generality that < cq < \d\ — 1. So, 
for ?j sufficiently small, 



J F s I u + iv,0; I ^ jj /i(u)du 



L f-^ „ co + md\ \d (co + md d 

0<|d|<A|c () + md| 

xxs((c + md)v 1 ' 2 ) m(by) r/ 2 (dy) + G\ T; (t; 1/2 logv), 

where ao = ao(b,d) and co = co(6, d) are chosen as above. We have dropped 
the restriction that gcd(6, d) = 1. 

For terms with \m\ > 1, we obtain upper bounds by observing 

1 1 

< 



| Co + rad\ (|m| — l)|d| ' 

and replacing the restriction imposed by X3 with the condition {\m\ — l)|d| < 
w_1 ^ 2 ( s 3 + fa)- For terms with m = 0, ±1, we have 

1 A 

< 



I co + md| |d| 

and we replace the restriction corresponding to X3 with \d\ < Av~ 1 / 2 {s^ + 83), 
since \d\ < A\co + md|. 

The restriction coming from \i means for d > 

sid - — - < b < (si + <5i)cZ - — -, 

cq + md cq + md 
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which we extend to 



A A 
s id- r^r <b < (s 1 + Si)d + T-77, 

\d\ \d\ 



and for d < 0, 



(s 1 + 8 1 )d — -<6<sid- 



cq + md Co + md 

which we extend to 

(s 1 + 5 1 )d-^<b<s 1 d + ^. 
We thus have (with n = \m\ — 1 for \m\ > 1, and n = 1 for m = 0, ±1) 

h(u)du 



/ Fi (" + i "' 0; (y)) 



<^5 2 v l/2 E jKm(by)r]2(dy)+Os,r 1 (v 1/2 logv), 
b,d,nez l nd l 

with the summation restricted to 

A A 

si\d\ - — < ±b < (si + <Si)|d| + rr, , n|d| < max(A, l)t>" 1/2 (s 3 + <5 3 ), n > 0. 
\a\ \a\ 

5.10.4. Since the components of (V, 1) are linearly independent over Q, 
Weyl's equidistribution theorem ([33, Satz 4]) implies that 

E Viipy) <Z\d\6i I r?i(x)dx, 

Sl |d|- ] A<±6<( Sl +5 1 )|d| + 1 ^ T 

uniformly for \d\ > w -1 / 4 large enough. For \d\ < -u~ 1//4 we use the trivial 
bound 

E m(by) = o StV {v- 1 ' A ), 

S1 \d\-^<±b<(s 1 +5 1 )\d\ + ^ 

for small enough v. Therefore 
J F 5 ^u + iv,0; ^ ^ h(u)du 

n>0 71 ^T* 

|d|<n-lu- 1 / 2 ( S3 +5 3 ) 

where the last term includes all contributions from terms with \d\ < i; -1 / 4 . 

5.10.5. We split the remaining sum over n into terms with < n < v^ 1 ^ 
and terms with n > v~ 1 ^. In the first case we have, for v — > 0, 

™ 1/2 E ^(dy) < (s 3 + I %(x)dx 

0<|d|<n-^- 1 /2( S 3 +< 5 3 ) Tfe 
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by Weyl's equidistribution theorem. For n > v one simply uses the trivial 
bound 

J2 V2{dy) <n- 1 «- 1 / 2 (s 3 + ( 53). 

0<|(i|<n- 1 t)- 1 /2( S3+( 53) 

5.10.6. We conclude 



limsup J F$ I u + iv, 0; ( ^ j j h(u)du 



h{ 1 

^ 6i5 2 (s3 + 5 3 ) / i7i(x)e£x limsup n~ 2 / r/ 2 (x)dx+ V" 



n- 2 



n<i)- 1 /4 n>u- 1 /4 

Since lim^o S n <u-V4 = x < oo and lim^o Z)n>«-i/4 = 0, the 
lemma is proved. □ 

5.11. Proof of Theorem 5.7. 

5.11.1. By Propositions 5.4 and 5.5, we find a convergent subsequence of 
p ti with weak limit v invariant under \E ,E . Hence for any bounded continuous 
function F on T\G k , 

lim p ti (F) = u(F). 

5.11.2. Following [17], we denote by H the collection of all closed connected 
subgroups H of G k such that TnH is a lattice in H and the subgroup, which is 
generated by all unipotent one-parameter subgroups of G k contained in H, acts 
ergodically from the right on T\TH with respect to the if-invariant probability 
measure. This collection is countable ([18, Th. 1.1]), and we call H* C H the 
set containing one representative of each T-conjugacy class. 

Because SL(2, R) k {0} and {1} k R 2k are each generated by unipotent one- 
parameter subgroups, so is G k , which of course acts ergodically (with respect 
to Haar measure jj) from the right on T\G k , and so G k € H. 

Let 

N(H) = {jeG^fJcg-H 
S(H) = |J N(H'), 

H'eH, H'CH, H'^H 

and 

T H = n(N(H)\S(H)), 

where tt is the natural quotient map G k -► T\G fc . We denote by uh the 
restriction of v on T#. Then, for any g G N(H)\S(H), the group g~ l Hg is the 
smallest closed subgroup of G fc which contains \£g and whose orbit through 
ir{g) is closed in (cf. [17, Lemma 2.4]). 
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For all Borel measurable subsets A C T\G k , the ^^-invariant measure v 
admits the decomposition (see [17, Th. 2.2]) 

u{A) = £ MA). 
Hen* 

Furthermore (see [17] for details), for any ^ R -ergodic component l of vjj, with 
i a probability measure, there exists a g € N(H) such that i is the unique 
g^i/g-right-invariant probability measure on the closed orbit T\THg. In par- 
ticular, if ^(vr(5(G fc ))) = 0, then v = (i (up to normalization). 

5.11.3. Let us suppose first that there is at least one H € H with vn / 0, 
whose projection onto the SL(2, R)-component is a closed connected subgroup 
L of SL(2,R) with L / SL(2,R) (compare Appendix B). Let A be the projec- 
tion of r onto its SL(2, R)-component. Since r n H is a lattice in H, A n L 
is a lattice in L. We can therefore construct a bounded continuous function 
F(t, <f>; £) = F(t, 4>) such that 

With F independent of £, we apply the equidistribution theorem for long arcs 
of closed horocycles [10], [15], which yields 

For the above subsequence (5.11.1) we find, however, 

lim Pu {F) = f Fdu, 

«->oo J 

which leads to a contradiction. We shall therefore assume in the following that 
L = SL(2,R). 

5.11.4. The most general form of a closed connected subgroup H, for which 
L = SL(2,R) and which contains a conjugate of \I/q, is (see Appendix B) 

fr = (l;^ )^o(l;-^o), = SL(2, R) x 

where ft is a closed connected subgroup of R 2fc (i.e., Q is a closed linear subspace 
of R 2fc ), which is invariant under the action of SL(2,R). Since SL(2,R) k {0} 
and {1} k d are generated by unipotent one-parameter subgroups, the same 
holds for Hq and hence for H. The right action of H on T\TH is obviously 
ergodic with respect to the (unique) //-invariant probability measure i, and 
therefore H E H. 
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5.11.5. Let us consider the orbit 

r\THg = r\T(l;Z )H g 
with g £ N(H) and thus g = (1; -£ )g £ N(H ). Note that 

Ho)h Ho)) _1 ^»^» 

for alH £ M, a € and 

H:))»«:)r-(-(.f))* 

The right-hand side is an element of f/o f° r all i £ R if and only if 



We therefore have the explicit representation 

a 




N(H ) = H 1; * : a £ R fe 



5.11.6. Let us suppose in the following that / for some H ^ G k , 
i.e., O 7^ R 2fc . We denote by B^{r) the open ball {x £ R k : ||x|| < r}. Then, 
for any r > 0, we define 

E(r) = r(l;Co)^o|(^l; ( o )) : a€B ^} 

= |(m ; £ + M^ a ^ : M£SL(2,R), £ £ a £ B fc (r)J , 

where O = r(£ + 17) is a closed subset in R 2fe . We fix r large enough so that 
the restriction of vn on T\S(r) is nonzero. 

5.11.7. Let us discuss the structure of Vt in more detail: Since V is of finite 
index in V = SL(2,Z) k l? k we see that r n H is of finite index in V n ff. 
Furthermore r (~l ii is a lattice in ii, and so V n ii is a lattice in if. Then 
clearly (1; — £ )r"(l;£ ) n Hq must be a lattice in Hq. With 

(1; -^ )r'(l; Co) = {(M; (M - 1)£ + m) : M £ SL(2, Z), m £ Z 2fe }, 

the lattice property in turn implies that (M — 1)£ £ Q + J? k for all M 
in a finite index subgroup A C SL(2,Z). The orbit SL(2,Z)£ /(ft + Z 2fc ) is 
therefore finite in R 2k /(n + Z 2k ); we denote by , ^ 2) , . . . , £ { J) } a finite set 
of representatives. With this, we conclude 

r>(t + n)= (j^ + n + z 2k . 
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The fact that (1; — £ )r"(l; £ ) n H is a lattice in H implies also that Z 2k n Q 
is a euclidean lattice in Q. Hence there is a compact fundamental domain 
fz 2k n£i c ^- We may therefore write 

J 

Note that ^Fz 2k nn ^ s a ^ so compact in M 2fc , since Q, is closed. 

We conclude by observing that r'(£ + Q) is, of course, a finite covering 
of 17, because T has finite index in V'. 

5.11.8. Consider the subset £<5(r) of S(r), given by 

S 5 (r)=T j(M;£ + M^ a ^ iMGPj, £ G fi, a G B fc (r)| , 

where is an open subset of SL(2,R) specified below. 
In the Iwasawa parametrization 3.8 

M _ ( uv~~ l l 2 sincp + v x l 2 cos uw -1 / 2 cos </> — u x / 2 sin (f> \ 
I t^ 1 / 2 sin</> v _1 / 2 cos0 i' 

we have 

E,(r) = r|(r > ^+( (u + ,, a 00t ^ )a )): 

(r,0) GP^e f2, aG B k (rv-V 2 am<f>)^ , 

where is now chosen to be the open set of elements (r, 4>) G SL(2, R) subject 
to the restrictions 

< u + v cote/) < 5, -1 < v~ 1/2 cos</> < 1, 1 < v~ l/2 sin^ < 2. 

For the set 

n 5 (r) = r I [tA; t + ( {U + ^ 0) * ) ) : (r, <P) G V s , i G S, a G B fc (r) } , 
we find 11,5 (r) C S<j(r) C n^(2r). Let us finally define 

= rj^-,0;C + £+ ^ :(t,<I>)€Vs, C € B 2 fc(e), ^fi, a€B fc (r)|, 

where i?2fc(e) C M 2fc is the open ball of radius £ about the origin. Thus, n^^r) 
is a full dimensional (but thin) open set, which contains ILj(r) if 5 > is chosen 
small enough. That is, for any e > there is a 5 > such that 

E*(r) C fl £ , 5 (2r). 
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5.11.9. The characteristic function of II := H ej g(2r) therefore satisfies 
Xfj( r 5 0;£) = 1 f° r an ( r ><^£) e S<5(r). Hence, and because ^|r\s(r) / 0, 
there is a constant q, > which is independent of £ and e, such that, for all 
£ > 0, (5 > sufficiently small, 

fj, /" /" dudv d(f) 
Uh(l\U) = / Xg > c b / cot 2 = 4q>0 

^ ^ — 1<D _ V 2 COS <^><1 ^ 

1<» -1 / 2 sin 4><2 

and so 

f(r\fi) > v H {T\fi) >4c b J. 

Since r\IT is open, we have along the subsequence fi,*2, ... in 5.11.1 (Theo- 
rem 1, p. 311 in [28]) 

liminfp ti ( Xfi ) > i/(r\fi) > 4c b c>. 

5.11.10. On the other hand, 

7 GSL(2,Z) 

where (as in 5.8) 

fs{T,4>) = xi(u + v cot (/)) X2{v~ 1/2 cos0) X3(v~ 1/2 sm(/)) 

and xi, X2 5 X3 are the characteristic functions of the intervals [0,5], [—1,1], 
[1,2], respectively. The function rj e is the characteristic function of the set 

{( C + * + ( a )) : CeB ^ £ ^ * G "' aG^(r)|+Z 2fe . 

By Lemma 5.9, there is a constant > which is independent of 5 and e, 
such that 

limsupp ti (xs) < c t 5 / 

for all sufficiently small e, 5 > 0. 
We conclude that 



4c b <c fl /" n e (£)d£. 



This contradicts our assumption that q, > 0, if we can show that the integral 
over i] £ tends to zero, as e — ► 0. We will check this by a dimension consideration. 

5.11.11. To this end we need to show that, if Q, ^ M. 2k , we have 
dim|U+ ( ° J J : ^gfi, aG^(r) < 2k. 
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In view of 5.11.7 this holds if and only if the dimension of the linear space 

V = n + W, = j ^ ° : a G K fc j , 

is strictly less than 2k. Suppose diml/ = k, and let bi, . . . ,h\ form a basis 
of SI. Then there exist vectors b^+i, . . . , b K G W such that bi, . . . , b K is a 
basis of V. Hence 

V = Q®U, U = span{b A+ i, . . . ,b K }. 
The linear subspace 

*•-(! ?m?) 

clearly satisfies U n U* = {0}, and also U* f]Q = {0} since U n £1 = {0} and 
Q is SL(2, K)-invariant. Hence 

V (BU* C R 2k , 

and so diml/ = 2k implies dim [7* = dim U = 0, which occurs only if f2 = V. 
Thus dim Q < 2k implies dim V < 2k and the claim is proved. 

5.11.12. Therefore vjj / if and only if H = G h , and hence the only limit 
measure of converging subsequences is the normalized [i. The uniqueness of 
the limit measure implies finally that every subsequence converges [28]. □ 

6. Diophantine conditions 

6.1. So far, all equidistribution results are valid only in the case of bounded 
test functions F. We will now extend these results to unbounded test func- 
tions F, which grow moderately in the cusps of T\G k . This will, however, only 
be possible under certain diophantine assumptions on y. 

6.2. To this end let us discuss the following model situation. Let G = G 1 
and r = SL(2,Z) k Z, 2 . Define furthermore the subgroup 

r °° = {( T ) :mG 4 CSL(2,Z), 



and put 



^ :=Im(7T) = RT^' for7= ( c d 



and 
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Let xr be the characteristic function of the interval [R, oo), 

XR(t) ~- 

For any / G C(R), which is rapidly decreasing at ±00, and (3 G J£, the function 



1 (t>R) 
(t < R). 



f r (t;z)= e E/((y7 + ^K /2 )<xi?K) 

7eroo\SL(2,z) mez 

is readily seen to be invariant under the action of T. If r lies in the fundamental 
domain of SL(2,Z) given by Fsl(2,1) = {t G f) : ti G [— |, |r| > 1}, and if 
furthermore R > 1, then -Fr(t;£) clearly has the representation 

*fl(r;o = e {/((y+^ l/2 )+ / ( ( - y+m)z;1/2 )}^ w - 

mez 

The sum over m is rapidly converging because / is rapidly decreasing at ±00. 
We note that Fr can alternatively be represented as 

F R (r;t) = E /((!)• (7C + n)^ 2 ) <X R K) 
( 7 ;n)ef 00 \r W / / 

with the abelian subgroup 



6.3. We will assume from now on that / > 0. The L 1 norm of Fr over 
r\G is then 

JT\G 

with Haar measure 

dudv d(j) dx dy 
dn{T,4>;£) = . 

Then 

AFr) = L f(yv 1/2 ) v P xr(v) dfi(T,<f>;S) 
Jr oa \G 

and so 

f /"oo c>— (3/2— /3) i- 

H(F R ) = 2ir / f(w)dw / v^' 2 dv = 2n — - / /(«;)<*«, 

JR JR 6/2 — p Jm. 

for /3 < 3/2, and ^{Fr) = 00 otherwise. Of special interest will be the case 
(3 = 1, for which 

/i(Fji) = 4^/T 1 / 2 / f(w)dw. 
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6.4. There is a well known one-to-one correspondence between the coset 
roo\r and the set 

{(0, 1), (0, -1), (1, 0), (-1, 0)} U {(c, d) € I? : c, d + 0, gcd(c, d) = 1}, 

given by 

( r, h \ 

-> (c,d). 



a b 
c d 



We may therefore write 

f r (t;H)=J2 {f{(v + m)v 1/2 ) + f{(-v + ™)v 1/2 )y X R(v) 



+ E / (*+ 



,1/2' 



m) 



+ f (~ x + m)- 



v 13 ( V 



+ E E / \( cx + d y + 



,V2 



m) 



|cr + d| y |cr + d\ 2 ? XR y\cr + d\ 



{c,d)e1 2 m&L 

gcd(c,d) — 1 
c,d#0 



Prom here on, we will only consider the case (3 = 1, and £ = \0, y). 

6.5. Proposition. Suppose h 6 C(M) zs positive and has compact 
support, and let y be diophantine of type k. Then, for any R > 1 and e,e' with 
< e < 1 and < e' < 



limsup / i^R n + if ; 

i>->0 ilul^ 1 " 6 V 







h(u) du = £:£ '(R 



where y, f and h are fixed. 

The proof of this proposition requires the following lemma. 

6.6. Lemma. Let a be diophantine of type n, and f G C(R) be rapidly 

decreasing at ±oo and positive, f > 0. T/ien, /or any /ixed ^4 > 1 and < e < 
l 

K-l> 



I 

d=lmGZ 



f:E/( T ( rf «+ m ))« 



1 (T £ <D< T^) 



uniformly for all D,T > 1. 
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6.7. Proof. 

6.7.1. Order a, 2a, . . . ,Da mod 1 in the unit interval [0, 1], and denote 
these numbers by0<</?i<...<</?£><l. Clearly <pj+i — ipj = kja mod 1 for 
some integer kj € [—D,D]; therefore, and because a is of type k, 

C C 

for some suitable constant C > 0. Hence in any interval of length I there can 
be at most 0(D K ~ 1 £ + 1) points. 

6.7.2. As to the first bound, take X[-R,R] to be the characteristic function 
of the interval [—R,R] with R > 1. Then 

D 



d=l m£Z 

for > R, since \da + m\ > > D ^-i ■ The argument in 6.7.1 shows 

that 

D 

E E X[-R,R] (T(da + m)) = 0(RD K ^ 1 T^ 1 + 1) = 0(R) 



d=i 

for D K ~ X T~ X < 1; hence 
D 



d=lm£Z k D K 1 ' 



i 



in the range D < T"- 1 . Since / is rapidly decreasing, we have for any I? > 3 

oo 

/(*)« E^xwK*)' 
i 

and hence, when D < T K ~ X , 

£> oo / n K - 1 \ B ~ 2 

ee/(^+h)« e v 

1 -, 

which proves the first bound in the range D < T £ < T"- 1 , for e < ^-j-j-. 

6.7.3. To prove the second and third relation, we follow [5, pp. 13-14]. 
Given any positive integer q (to be fixed later) divide the sum over d into 
blocks of the form 

b+q-l q-1 

E E f(T(da + mj) =E E f(T(ba + da + m 

d=b meZ ' d=0meZ 
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(The last block might contain less than q terms, but this is irrelevant since we 
are seeking an upper bound.) There are 0(^ + 1) such blocks. Take a rational 
approximation £ to a with \a— || < q~ 2 and p, q coprime, then the above sum 
is 



£ E/ ( T ( fa+ *±p +m )) 



d=0meZ 

Since dp runs through a full set of residues mod q, the above equals 

E E / (T (ba + T -^^ + m))=J2f (^(Qba + r + O(l))) . 

r=0mgZ V V q 77 reZ Vy 7 

The term (76a may be replaced by the nearest integer +0(1), and so 
]T / (^-(qba + r + 0(1))) = ]T / (^(r + 0(1))) 

which in turn is clearly bounded by 0(Jp + 1) for / is rapidly decreasing. 



Therefore 

D 



EE/( r (*» + m))<(£ + l) (| + l). 

6.7.4. By Dirichlet's theorem, we may take | such that q <T and |a— 1| < 
(7 _1 T _1 . Since a is of type k, we have Cq~ K < \a — ||, so that 



and finally 



££/(r(da + m))«-§- + l. 



□ 



6.8. Proof of Proposition 6.5. 

6.8.1. Because we are only concerned with upper bounds, we may assume 
in the following without loss of generality that / is positive and even, i.e., 
/ > 0, f(-w) = f(w). 

It follows from the expansion in 6.4 that, for v < 1, the first term is absent, 
since Xr( v ) = (recall: R > 1); hence we are left with 



+ 2 E E/((^+™)- 



(c,d)e2 

gcd(c,d) = l 
O0,d^0 



cr + d| y |cr + d| 2 \|cr + cZ| 
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6.8.2. As to the first term in the above expansion, a simple change of 
variable u = vt shows that 

L» 2 S'("¥) w XR {w) h{u)du 

= 2 L^J U^™ W+l XR (opTT)) ft(l,1)<i( 

/" (it 
< /> /l / , 7TTT' 

J|t|>-u- £ I + 1 

since the sum over m is converging uniformly with respect to t and v due to 
the fact that v(t 2 + 1) < i?" 1 < 1. 

For e > the value of the above integral converges to zero as v — ► 0. 

6.8.3. We obtain an upper bound for the remaining terms, by dropping 
the condition \u\ > v l ~ £ in the integral. We are thus led to estimate 

S i v ) = E E J{v,c,d,m) 



gcd(c,d) — 1 



with 



J(v ' c ' d ' m)= U [ {dy + m) WTd\) WTW XR (r?) Ku) d " 

We substitute t = v _1 (u + |) for u, yielding 



c 2 v(t 2 + 1) / c 



The range of integration is bounded by 



1 1 

-ft < n /.n 1-e., t <C 



cM* 2 + l)' ' ' cyfcR' 

This implies |vt| <C f 1 / 2 ^ 1 /? -1 / 2 is uniformly close to zero, and hence, because 
of the compact support of h, we find \d\ < Mc, for some constant M > 
depending only on the support of h. Therefore 

oo 

c=l 0<\d\<Mc m£Z 



with 



XR ( cMt 2 + i)J dL 
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6.8.4. In order to apply Lemma 6.6 with D = Mc, T = (c 2 v(t 2 + 1)) 1 / 2 > 
\fR > 1, we split the t-range of integration into the ranges 



(1) 
(2) 
(3) 

which correspond to 



Mc < {c 2 v{t 2 + 1))~ £ ' /2 

{c 2 v(t 2 + l))- £ '/2 < Mc < (c 2 v{t 2 + 1))-^) 
Mc > (c 2 v(t 2 + l)) - ^ 1 ), 



(1) 
(2) 
(3) 



D <T £ 

T £ ' < D < T^r 
D > T^r. 



Here, e> < ^. 

We denote the corresponding integrals by K\ (v, c, d, m) , K2(v, c, d, m) and 
K 3 (v,c,d,m), respectively. 

6.8.5. Because R' 1 / 2 > T -1 , 
E E E gi(t>,fid,m) « ^~ A/2 E^ L ^T** ( C 2^2 + n 

O00<|d|<Mc meZ c>0 C A 1 ) 1 +i V c ^ r + -U 

^ c 2 J t 2 + l 



dt 



dt 



oo 



« ir A / 2 . 



6.8.6. In order to obtain an upper bound, we can relax the second range 
T 6 ' < D < T^r to R £ 'l 2 < D, since R 1 / 2 < T. This yields 

E E E c > d > m ) « E c ' 2 [ 72TT dt R ~ £ ' /2 - 

c>0 0<\d\<Mc meZ ,,.^r,ri/o. ' ' 1 



Mc>R s 72 



6.8.7. In the third range, we find (putting <5 = ^zj), 
E E E #3(v,c,d,m) 

c>0 0<|d|<Mc m&L 

f-i C J 7(3) Vc 2 V(t 2 



+ 1) 



< E 



c>0 



f(t 2 + 1)^X1 

JR 



c 2 v(t 2 + 1) 



dt 



+ 1) 



( t 2 + 1) f-l^ 
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For the inner sum there exist the upper bounds 



.z 2 v(* 2 + 1) 

= [t,(t 2 + l)]-" 2 J~ x^R (1) dx = [v(t> + I)]- 5 / 2 H 



and so 

p-<5/2 /■ „ra-5/2 

EE E%M,m)<V /(* 2 + i)- 1 cft = ^=— . 

OO d<c mez JM 
The proof of Proposition 6.5 is complete. □ 



7. Equidistribution and unbounded test functions 

7.1. Let us define the characteristic function on T\G k (cf. the proof of 
Proposition 5.4): 

Xr(t) = XiiK)> 
7e{r 0C u(-i)r 00 }\SL(2,z) 

where is the characteristic function of [R,oo). 

7.2. We shall consider functions on T\G k , which grow moderately in the 
cusps. To be more precise, we will require that, for some fixed constant L > 1, 
the function F is dominated by Fr; that is, for all sufficiently large R > 1, 

\F( T ,<i>;t)\X R {T)<L + F R (T;t) 

uniformly for all (r, £ G fc . The function Fr(t;£) is now viewed as a 
function on G k (rather than G 1 as in Section 6); that is, for 

£ = \xi, ...,x k ,yi,.. .,y k ) 

we put 

Fr(t;£) E E /((i/1.7 + m H /2 ) VyXR^) 

7er OQ \SL(2,z) mez 

which is invariant under SL(2, Z) k Z 2fc (Section 6) and thus also under T. 
Note that Fr(t; £) is constant with respect to X2, . . . , x k and 7/2, • • • , y&- Again, 
/ G C(R) is rapidly decreasing at ±00, positive and even. 

7.3. Theorem. Let T be a subgroup of SL(2, Z) k Z 2fc of finite index. Let 
h be a continuous probability density R — > R + mi/i compact support. Suppose 
the continuous function F > is dominated by Fr. Fix some y 
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the components of the vector (V, 1) G M. k+1 are linearly independent over Q. 
Then, for any e with < e < 1, 

lim inf / F [ u + iv, 0; ( ^ ) ) h(u) du > — ^ —. , / F da. 

w-o J| u |>„i-e ^ V y / / ~ Mr\G fc ) ir\G* ^ 

Assume furthermore that y\ is diophantine. Then, for any e with < e < 1, 

lim sup / F ( u + iv,0; ( ° | | /i(-u) tin < — — - —, . / F du. 

w -o J\u\>vi-c \ \yJJ ~ fi(T\G k ) J r \G k 

Proof. We obtain the lower bound from the function 

Gr(t, 0; £) := F(r, 0; £) (1 - X fl (r)) < F(r, </>; £). 
Clearly, Gr is bounded. Therefore 

/ G R (u + iv,0;£) h{u)du = / G R {u + iv, 0; £) /i(u) + O r (v 1_£ ), 

and, by Theorem 5.7, 3 

lim/ G R (u + iv,0;£) h(u)du= , / Gijdu. 

f->o «(1 \G k J Jr\G fc 

Now since < FXr < LXr + Fr for large enough we have 

/ FX R dfi< I (LX R + Fr) du « Li?" 1 + i?- 1 / 2 
Jr\G k Jr\G k 

from 5.4 and 6.3, and hence 

/ G R dfi= ( Fdfi + OiLR- 1 + FT 1 / 2 ). 
Jr\G k Jr\G k 

In summary 

lim inf f F(u + iv, 0; £) h{u) du > 1 / Fdu + 0(iT ^ 

for all i? large enough. The assertion on the lower bound follows now from the 
fact that R can be chosen arbitrarily large. 

For the upper bound, notice that for R large enough, 

F(r, <f>; < F(r, <f>- £)(! - X fl (r)) + LX fi (r) + F fi (r; 0- 



3 The fact that Gr is only piecewise continuous should not worry us: Theorem 5.7 can easily 
be extended to such functions by approximating these from above and from below by continuous 
functions. In any case, the argument presented here works as well if \R ls smoothed slightly, which 
makes Gn continuous. 
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By virtue of the bound obtained in the previous paragraph, and by Proposi- 
tion 6.5, we find that 

limsup / F(u + iv, 0; £) h(u) du 

H{1 \G /t ) Jr\G k 

for some small constant r] > 0. This holds again for arbitrarily large R, and 
the statement is proved. □ 

7.4. Corollary. Let T, h, y be as in Theorem 7.3, and F : T\G k — »■ C 
be a continuous function which is dominated by F R . If y\ is diophantine, then, 
for any e with < e < 1, 

lim / F I u + iv,0; ( ° | | h(u) du = — — ; . / F da. 

^oJ H>v i-e \ \yJJ fi(T\G k ) J r \G* 

Proof. Define 

' Re F(r, <f>; £) if Re F(r, <j>; £) > 0, 



Re+F(r,0;£) 



if Re F(t, </>;£) < 0, 



and Re_ F = Re + F— Re F. We similarly define Im± F as the positive/negative 
part oflmF. Then 

F = Re+ F - Re_ F + ilm+ F - ilm_ F 

with 

< Re+ FX R <L + F r , < Re_ FX R < L + F R , 
< Im+ FX R <L + F r , < Im_ FX R < L + F R . 
We can thus apply Theorem 7.3 to each term separately, 

lim f Re±F f u + iu,0; I ° ) ) h(u) du = , } . s [ Re±Fdri 

and likewise for Im-t F. □ 

7.5. Since in our main application T = T k , which is a subgroup of finite 
index in SL(2,Z) x (±Z) 2k rather than in SL(2,Z) x Z 2fc (Lemma 4.12), we 
restate Corollary 7.4 in the following equivalent way. Define the dominating 
function F R on T\G k by F r (t;£) = F r (t; 2£), with F R as in 7.2. 

7.6. Corollary. Let T be a su bgroup o/SL(2,Z) x (±Z) 2fc o/ /mite 
index, h, y 6e as in Theorem 7.3, and F : — > C a continuous function 
which is dominated by Fr. //yi is diophantine, then, for any e with < e < 1, 

lim / F ( -a + if , 0; ( ° ] ) /i(u) = /T ^„,,, / F d/x. 

«-»o./ H>t ,i- e I \y u{T\G k )J v \G* 
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Proof. Apply Corollary 7.4 with the test function F : T\G k — > C defined 

by 

F(T, ( f>;H) = F(T, ( f>;^) 

where 

'-((::)")'((»:)*) 

is a subgroup of finite index in SL(2, Z) k Z 2fc (compare Remark 4.13). □ 



8. The main theorem 

8.1. Main Theorem. Suppose f(wi, w 2 ) = i>i{w\ +w?,) andg{w\, w 2 ) = 
ip 2 (w 2 + w 2 ) wit/i Vi j ^2 £ <S(R+)- iet h be a continuous function R — > C u>ii/i 
compact support. Assume that , 3/2 , 1 ore linearly independent over Q and 
that yi is diophantine. Then, with £ = *(0, 0, 3/1, 3/2)5 

lim / 6f (u + ru, 0;^)6 g (n + if,0;£) h(u) du 

= 7r|2vr/i(0) + J h(u)du} J tpi(r)ip 2 (r) dr. 

The proof of the main theorem requires the following two lemmas. 

8.2. Lemma. If f,g e S(M. 2 ), 

H(T 2 \G 2 ) L*\G 2 0/( ' r ' ^ >; ^ & s( T ^ ( f ) ^) d t i = J J f(w 1 ,w 2 )g(w 1 ,w 2 )dwi dw 2 . 

Note that if f(w\,w 2 ) = tpi(w 2 + w 2 ) and g(wi,w 2 ) = i/) 2 (w 2 + ^2), then 

J J f(wi, w 2 )g(w i,w 2 )dw\dw 2 = it J ipi(r)tp 2 {r) dr. 
Proof. A short calculation shows that 

J e/(T,0;£)6 9 (T,0;£)(i£ = J J f^(w 1 ,w 2 )g 4> (w 1 ,w 2 ) dwi dw 2 . 

Since = R(i, <j))f with R(i, <j>) unitary, we have 

J J f<f,(wi,w 2 )g ( f > (wi,w 2 )dw 1 dw 2 = J J f(wi,w 2 )g(w 1 ,w 2 )dw 1 dw 2 . □ 
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8.3. Lemma. Suppose f{wi,w 2 ) = ipi(w 2 + w?,) an< ^ 9{ w i-> w 2) 
ip2(wi + wi)i w tth ^1^2 S <S(M+). For any \ < 7 < 1, 



lim / 9 f ( 

^^0j| u | <t ,7 y 



u + if, 0; 



g u + ro,0; 



I h(u) du 



POO 

= 2vr 2 /i(0) / ^i(r)^(r)dr. 
•/ 

Proof. Proposition 4.11 tells us that 



/ 1 --> ar g r ; 



Q g ,argr; 



^/argr(0,0)< 7argr (0,0)+O i? 



mi 



holds uniformly for Im(— r 1 ) = v\t\ 2 > \. This condition is met, e.g., when 
\u\ < v 1 ! 2 < 1. For \u\ < v 1 < 1, with | < 7 < 1, the error term is bounded 
by 



Or 



Or{v r ^). 

Now replacing (101,102) by polar coordinates (r cosC,"rsin£) yields 

/argr(0,0)5r argr (0,0) = | J J e ^ \ {w\ + w\ ) ^ / (lOi , W 2 ) tZwi tfoi> 5 



yy e ^(tof + ^2)^ g(wi,w 2 ) dwi dio 2 | 

-71 



(ri - r 2 )u 
2v 



i>i{ri)i> 2 {r 2 ) dr\dr 2 



-7T 2 -0l 



^2 



where if) denotes the Fourier transform 

POO 

tf>(u) = / e(ur)ip(r) dr. 
Jo 

Clearly $ G L 2 (R) for V G 5(M+) C L 2 (M + ). Thus, 



s it + iu,0; 



I /t(u) du 



- W M <>(£)M£)*" ) * +o " ( ' 1H ^ ,,) 

= 2tt 2 ( 4> 1 (u)4> 2 (u)h(2vu) du + R (v^ +R ^- 1} ). 

J2\u\<vi- 1 
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Since h is continuous, for any given e > we find a vq > 0, such that 
\h{2vu) — /i(0)| < e, uniformly for all 2\u\ < v 7_1 , < v < vq. 
Thus for any e > 



£S 0/ ( u + ° ; ( y ) ) 09 (" + ° ; ( y ) ) Ku) du 

= lim 2vr 2 {/i(0) + 0(e)} / ^(u^iu) du 

v^O J2lul<i)T- 1 



= 2tt 2 {/i(0) + O(e)} / ^i(w)^ 2 (n)du 

= 2tt 2 {/i(0) + 0(e)} / ViMMO^r 
Jo 



by Parseval's equality. Because e > can be arbitrarily small, the claim is 
proved. □ 

8.4. Proof of the main theorem. 

8.4.1. Due to the linearity in h of the integrals in 8.1, we may assume 
without loss of generality that (i) h is positive (compare the argument used 
in the proof of Corollary 7.4) and (ii) that h is normalized as a probability 
density. 

8.4.2. Let us split the integration on the left-hand side of 8.1 into 



+ 

\u\<v 1 ~ s ^lu^^ 1 

for some small e > 0. The first integral gives, by virtue of Lemma 8.3, the 
contribution 



2n 2 h(0) / ^i(r)^2(r)dr. 
Jo 



8.4.3. In order to apply Corollary 7.6, we need to construct a function Fr 
of the form studied in 7.2, which dominates \F\. Let us define 

f*(wi) = sup sup \f < j>(w 1 ,w 2 ) g <j> (w 1 ,w 2 )\, 

w 2 <=R 4><=R 

which is clearly rapidly decreasing at ±oo since, for every T > 1, there is a 
constant ct > such that 

f*( Wl ) < sup c T (l + Jwl + wi] < or(l + H|)~ 2T , 

W2&L \ / 

holds (cf. Lemma 4.3). 

Choosing (compare 7.2) 

Fr{t;Z)= J2 E /* ( - \ (3/1,7 + m H /2 ) v i XR(v y ), 

76r oo \SL(2,Z) m6Z 
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we have for all v > R 

Fr(t; = F R (r; 2£) = v £ {/* ((f - « 1/2 ) + /*((? + Vi) v 1 / 2 ) } ; 

m£Z 

that is, 

^(r; = v {f ((§ - yi ) v^) + f ((-§ + yi) v 1 ' 2 ) } + 0(^- T ), 

for all yi G f + [— j, |], n G Z. By construction, for n = ^711,712), 



/ ((n - y)^/ 2 ) ^ ((n - y)^ 1 / 2 j < /* ( ( ni - yi > 
which implies that, for all v > R, R large enough, 

E A((m- y y/ 2 )„((m- y y/ 2 ) 

= v U f(n - yV /2 ) g* ((n - y)^/ 2 ) I + 0(0 



,1/2 



< vf* ((ni - yi)^ 1/2 ) + 0(i-- T ) = w E /* ((mi - yiy /2 ) + 0( 



mi 



uniformly for y = \yi, 2/2) G n + [|, ^] 2 , n G Z 2 . 

Therefore, by virtue of Proposition 4.10, we have, for all sufficiently large R, 

|e / (r,</»;Oe fl (r,0;OI<l + « £ /* ((f - ^ 1/2 ) < 1 + *h(t; £) 



m even 



for v > R, and so |0/0 s |Xr < 1 + Fr. We can now apply Corollary 7.6, and 
thus obtain the second term on the right-hand side of 8.1 (recall Lemma 8.2). 

□ 

8.5. Proof of Theorem 2.2. Recall that 



6/ (u + ij,0; Xo,O,a,0)J Q g (u + ij,0; \0,Q,a,/3)j h(u)du 

= nR2(ipi,ih, h, A). 
We have furthermore 

h( s ) = / h(u)e(\us\ du, h{u) = \ 1 h(s)e(—\us\ ds; 

hence 2h(0) = f h(s)ds and / h(u)du = h(0). □ 
8.6. Proof of Theorem 1.8. 

8.6.1. Let x[o, b] be the characteristic function of the interval [a, b\. Given 
any e > 0, we approximate x[°>fr] horn above and below by functions x± £ 
C°°(R) with compact support so that 

X-(s) <x[a,b](s) <x+{s), / (x+(s)-x-(s))ds <e. 
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Put 



h±(s) = x±{s) ± 7—^: 
1 + s z 



where 5 > is chosen such that 



Then 



45 



l + s 2 



ds < £. 



s s 



JM. \ 



s) - h-(s) + 



25 



l + s< 



ds < 2e. 



The inverse Fourier transform 



h±(u) = \ J h±{s)e ^— \us*j ds 



is continuous on R, infinitely differentiable on M — {0} and decreases, together 
with its derivatives, rapidly at ±00. 

8.6.2. We fix a smoothed characteristic function x £ C°°(IR) of compact 
support in [—2, 2], with < x < 1 an d x( u ) = 1 H u £ [—1,1]. Define 



h T ,±(u) = h±(u)x(jp) , 



which is continuous and has compact support in [— 2T, 2T\. For the Fourier 
transform 

hr,±( s ) = J R h T,±( u ) e (l us ) du 
we have, for some constant C, 

\h±(s) - h T ,±(s)\ < j IM«)I 1- X (| 



C 

du < I \h±(u)\ du < — , 

'\u\>T 1 



and (integrate by parts twice) 

1 



\h±{s) - h T ,±(s)\ < 



+ 



J"2 



/4(«)x' 



du 



du 



< 



C 



Therefore we find some T > 1 such that 



\h±(s) -h T ,±(s)\ < 



1 + s 



2 ' 
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Hence 



h T ,-( s ) < X[a,b](s) < h Tt+ (s), [ (h T ,+ ( s ) ~ h T ,-(s))ds < 2e. 

8.6.3. We will assume in the following that Y>i,Y' 2 > 0. Then 
1 \ - , / A, \ , / Ai 



7rA i*k 



J> (y) ^ (y) ^,-(A, -A fe ) 



^Z>$)*(t)^-*>- 

The functions /it,± satisfy the assumptions in Theorem 2.2, so the limits of 
left- and right-hand sides exist, and differ by less than 



3C 



o 



fpi (r)ip2(r)dr 



2ns 

for arbitrarily small e > 0. Hence 

£So ^ S ^ (y ) ^ (y) X[a, &](Aj - A fc ) = vr(6 - a) / ^(r)^(r)dr. 

8.6.4. Analogous arguments allow us to replace first and then ^2 by 
characteristic functions. □ 

For detailed discussions of approximation functions of the type used above, 
see [29] and references therein. 

9. Counterexamples 

9.1. Put 

Q a> p(m, n) = (m- a) 2 + (n - f3) 2 . 
For (a, (3) E Q 2 we find (see Appendix A. 10 for details) for A — > oo, 

R( a >fl[0,0] = — #{(mi,m 2 ,ni,n 2 ) G ^ 4 : (mi,m) / (m 2 ,n 2 ), 
< A, Qa,/3(mi,ni) = Q a ,/3(m 2 ,n 2 )} ~ c Qj/3 log A, 

for some constant c a ^ > 0. This fact will be the key in proving the first half 
of Theorem 1.13. 

9.2. Proof of Theorem 1.13 (i). Enumerate the rational forms Qa 3 ,f3j with 
(aj,(3j) € Q 2 as Pi, P 2 , P3, Because of the asymptotics 9.1, given any 
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A > 1, there exists an Mj > A such that 

#{(mi,m 2 ,m,n 2 ) e Z 4 : (mi,m) / (m 2 ,n 2 ), 



vrM,- 



Pj(mi,7ii) < Mj, Pj(mi,ni) = Pj(m 2 ,n 2 )} > 



log Mf 



log log log Mj 
Now since 

Q ai/3 (77ii,7ii) - Qa,p( m ^ n ^) = Qa.^imum) - Q < a . ^. (m 2 , n 2 ) 

+ 2(c*j - a) (mi - m 2 ) + 2(/3j - /3)(ni - n 2 ), 

we have that 

4 a ' /3) [-a,a](M J ) > 4 Q - ft) [0,0](M,) 

when la - a,| < —^=2 and \8 - 8A < -. Denote by Bj C T 2 the 

open set of such (a, 8). 

To summarize, given any A > 1, there exists an Mj > A such that 



R?' P) [-a,a](Mj)> 



log log log Mj 

for all (a, /3) G Pj. Individually, the sets Bj shrink to a point as A — > oo. Note, 
however, that for every fixed A the union 

U B 3 

j:Mj>\ 

is open and dense in T 2 , and therefore 

oo 

B=f] (J Bj 

A=l j:M 3 >\ 

is of second Baire category. 

So if (a, 3) € B, then, given any A > 1, there exists some M > A, such 

that 

P^[-a,a](M)> |° g f □ 
2 1 > jv y - i gi gi gM 

Note that the proof remains valid if log log log is replaced by any slowly 
increasing positive function v < log log log with v(M) — > oo (M — > oo). 

9.3. Proof o/ Theorem 1.13 (ii). By virtue of Theorem 1.8, there exists 
a countable dense set { (£j , (j ) G T 2 : j G N} for which the pair correlation 
density of the forms Oj := Q( jt Q is uniform. That is, for any A > 1, we find 
some Lj > A such that 

2vra - \ < Pj^'' 0) [-a, a](Lj) < 2na + \. 

A A 
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Let Aj C T 2 be the open set of (a, (3), which satisfy \a — £j| < Ej and \(3 — Q\ 
< Ej, where Ej > can be chosen in such a way that 

2 2 
2vra - - < R^' P) [-a, a](Lj) < 2vra + - 
A A 

for all (a, (3) £ Aj. Now, 

oo 

^=n u a j 

A=l j:L 3 >A 

is again of second category We conclude that if (a, (3) £ A, then, given any 
A > 1, there exists some L > A, such that 

2 2 
2vra-- <R^ P) [-a, a] (L) < 2vra + -. □ 
A A 

9.4. Conclusion of the proof of Theorem 1.13. Since A and 5 are of second 
Baire category, so is the intersection C = A n B. □ 



Appendix A. Symmetries 



A.l. We have seen in Section 5 that in the case when a, (3, 1 are linearly 
independent over Q, 



/ 



F(u + iv, 0; *(0, 0, a, (3))h{u)du 



converges for all suitably nice test functions F on T 2 \G 2 to the average of 
F over T 2 \G 2 , as v — > 0. This is no longer true when a, (3,1 are linearly 
dependent over Q, i.e., if we find integers (k, I, m) 6 Z 3 - {(0, 0, 0)} such that 
ka + 1(3 + m = 0. One of k, I must be nonzero, and we will assume in the 
following (without loss of generality) that I / 0, i.e., (3 = — j(ka + m). 

A. 2. Suppose a ^ Q. For any given function F G C(G 2 ) which is invariant 
under the left action of T 2 , we define a function F G C(G 1 ) by 



Fir, 



X 

y 



V 



— j(kx + m 

y 

V -)(ky + m) ) 



Since F is invariant under the left action of the subgroup 

" i o N 



( 7 ,n) G SL(2,Z) k Z z : 7 



1 



mod 21, n = mod 2/ } C T 1 , 
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we can identify F as a function on T^G 1 . The congruence group is of 
finite index in SL(2, Z) x Z 2 , and hence T^G 1 has finite volume with respect 
to Haar measure. 

A. 3. If a = - and 3 = 1 are rational, we define instead 

q * s ' 

F(r,^)=F(r,^X0,0,2,§)) 
which is a function on G° invariant under the left action of the subgroup 



= \ 7 e : 7 = I - I mod 2gs 



1 



Again, r^^C? has finite measure. 

A. 4. Example 1. Consider the case a = /? ^ Q. In order to remove the 
two-fold degeneracy we consider the symmetry-reduced set 

{(m — a) 2 + (n — a) 2 : (m, n) G Z 2 , m > n} 

whose elements we label by Ai < A2 < • • •• The pair correlation function of 
this sequence is now again Poissonian: 

A. 5. Theorem. Assume a = 3 is diophantine. Then 
lim R 2 [a,b](X) = J(6-a). 



Notice that the mean density is now | since we count only distinct ele- 
ments. 

A. 6. Sketch of the proof. The smoothed correlation function is 
2 

7rA 

(mi ,ni)££, A (m-2 ,n2)G^ z 

' (mi - a) 2 + (ni — a) 2 



iWi.^.M) = 4 E E 



x Vi 



X ^2 



A 

'(m 2 -a) 2 + (n 2 -a) 2 * 



a y 

x h((mi — a) 2 + (ni — a) 2 — (m2 — a) 2 — (n 2 — a) 2 )- 
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This is asymptotic for large A: 

d i i i u \\ 1 ( (mi - a) 2 + (ni - a) 2 

R 2 {ipi,ip 2 ,h,\) ~ ^ ^ ^ I X 

(mi,ni)6Z 2 (m2,n2) """ 



x ^ 2 ^ ~ ») 2 + ^ 2 ~ ^ 

xh((mi — a) 2 + (ni — a) 2 — (m2 — a) 2 — (ri2 — a) 2 ), 

since the diagonal terms m± = n\ or m 2 = n 2 give lower order contributions. 
The right-hand side of the above expression is equal to 

J 6/ (u + ij,0; \0,0,a,a)j Q g (u + i-^,0; \0,Q,a,a)j h(u)du. 
The corresponding test function 

F(t, 4>; \x, y)) = 8/(r, 0; x, y, y))6 9 (r, 0; V, a;, y, y)) 

is a function on r 1 \G 1 ; compare A. 2. Starting from Theorem 7.3 we can 
apply the same string of arguments as before. The only main difference is that 
Lemma 8.2 has to be replaced by the one given below. This yields (compare 
the main Theorem 8.1; we assume here that ipi,ip 2 are real- valued) 

lim f e f {u + w,0; \0, 0, y, y))O g (u + iv, 0; \Q, 0, y, yj) h(u) du 
= 2tt ^nh(0) + J h(u)du^ j ipi(r)ip 2 (r) dr, 

and hence 

lim Ra(ih,ih,h,X) = U(0) + J / h(s)ds\ [ Y>i(r)Y> 2 (r) dr, 
as needed. □ 
A. 7. Lemma. Iff,g£S(R 2 ), 

^T T V2 T j Xi\ G i 0/ ^ r ' ^' x ' y ' y ^ s( T ' ^ ; X) y ' y )) ^ 

[/(iwi, w 2 )g(w 1 ,w 2 ) + f(w 1 ,w 2 )g(w 2 ,w 1 )^ dwidw 2 . 

When f(wi,w 2 ) = ipi(w 2 + w 2 ) and g(wi,w 2 ) = ip 2 (w 2 + w 2 ), this yields 
J f{f(.wi,w 2 )g(wi,w 2 ) + f(w 1 ,w 2 )g(w 2 ,w 1 )}dw 1 dw 2 = 2ir ip ± (r)ip 2 (r) dr; 
compare 8.2. 
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Proof. Consider the function 



F {i~,4>) = \x,x,y,y))e g (T,^); \x,x,y,y)) dx dy 




This function may be viewed as a function on SL(2, Z)\ SL(2, R). By virtue of 
Proposition 4.10 one finds that asymptotically in the cusp (v — > oo) we have 



It follows from the classical equidistribution of closed horocycles [24] , [6] in the 
case of unbounded test functions (cf. Proposition 4.3 in [13]) that as v — > 



converges to the left-hand side in Lemma A. 7. The right-hand side of the 
above equation can, however, be worked out straightforwardly: The series 
representation of 6^ gives a natural Fourier expansion with respect to x and 
u. The zeroth Fourier coefficient, which we want to calculate, is given by those 
summands for which 



whose only solutions are obviously (mi = mi^n\ = 71-2) or (mi = 712, m2 = n-y). 
In the limit v — > 0, the zeroth Fourier coefficient is now easily seen to converge 



F{t,4>) = v 1 ' 2 / f (t> (w,w)g <t> (w,w)dw + R (v- K ). 




Q f (u + iv,(j); \x,x,y,y))e g (T, <f>; \x,x,y,y)) dx dy du 





to the right-hand side in Lemma A. 7. 



□ 



A further special case of interest is the following. 



A. 8. Example 2. When j3 = or (5 = ^ we consider the symmetry-reduced 
sequences Ai < A2 < ■ ■ ■ given by the sets 



{(m - a) 2 + n 2 : (m, n) GZ 2 , n> 0} 



or 



{(m 



a) 2 + (n - \) 2 : (m, n) GZ 2 , n > ±}, 



respectively. 



A. 9. Theorem. 



Assume a is diophantine. Then 



lim R 2 [a,b](X) = -(b-a). 

A^oo 2 
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The proof of this theorem is analogous to that of Theorem A. 5. 

A. 10. Example 3. If a = |, (3 = ^ are both rational, the integral 

/ F(u + iv,0)h(u)du 

of the corresponding test function 

F(r, 0) = @ f (r, <f>; \0, 0, f , § )) 6 9 (r, 0; \0, 0, f , §)) 
is diverging as v — ► 0; one finds in particular that in this limit 

1 f 2 i s ~ _-, 
— F(u + iv,0)du ~ b a plogv 
2qs Jo 

for some constant b a ,/3 > 0. This follows from arguments analogous to those 
given in [13, Th. 6.1]. 

Therefore, for A —>■ oo, 

^2[0,0](A) = -\#{(m 1 ,m 2 ,n 1 ,n 2 ) G Z 4 : (mi - a) 2 + (m - f3) 2 < A, 

(mi - a) 2 + (m - (3) 2 = (m 2 - a) 2 + (n 2 - /3) 2 } ~ c Qj p log A, 

for some constant c a ^ > 0. In the case a = (3 = this yields, of course, 
Landau's well known result on the asymptotic number of ways of writing an 
integer as a sum of two squares. 

Appendix B. Closed connected subgroups of SL(2,R) x M. 2k 

B. l. Suppose H is a subgroup of G k = SL(2,R) x M. 2k . Then 

H = {(Af;£) G G k : M G L, £ G C(M)} 

where L is a subgroup of SL(2,M) and C(M) is a family of sets, which are 
suitably chosen such that H is a group, but are otherwise arbitrary. 

B.2. Clearly = C(l) is a subgroup of R 2k , because (1;^)(1;^') ±1 = 
(1; £ ± £') for all €Sl implies ^±^'efi. 
Moreover, 

(M;0(l;O(M;0 -1 = for any (M;£) G tf, 

says that if £' G then M£' G hence is invariant under the action of L. 
This means also that {1} k 17 is a normal subgroup of H. Thus if (M; CC^O) 
is a set of representatives from the coset ({1} x ft)\H, 

(M;C(M))(M';C(M')) = (l,(i(M,M'))(MM';C(MM')) 

with cocycle 

<t(M, M') = C(M) + MC(M') - C(MM') G fi. 
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We choose C(M) in such a way that C(l) = 0. 

B.3. If H is a closed connected subgroup of G k , then L is a connected Lie 
subgroup of SL(2,R). Since all such subgroups are closed in SL(2,R), L is a 
closed connected subgroup of SL(2,R). 

B.4. Let us assume in the following that the subgroup 

•?=((; ?H 

is contained in H, and that L = SL(2,R). Then 
and thus (R; £ ) G H for some vector 

xo \ 



yo 



Since conjugation by 



9 



Li Xo : yo 



' 2 







yields 



and 



for any t G R, we may assume without loss of generality that xo = yo (replace 
H with g^Hg). 



Note that 



,M: -hi l 



is in H, and so is the conjugate 

Hi))((!:)-)Hc))-((;:).(T 

This implies, however, that 
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for all t £ R, and so 

Ht))H:))Ht)H°>- 

Because the elements 

(; :)<-»>• (° • ) 

generate SL(2, R), we find trivially that ^ and (R; 0) generate SL(2, R) k {0}, 
and thus H = SL(2,R) x SI. 

B.5. Since f2 is invariant under the action of SL(2, R) and H is closed and 
connected, it is a closed connected subgroup of R 2fc , i.e., VL is a closed linear 
subspace. 

B.6. We conclude that any closed connected subgroup H of G k , for which 
L = SL(2, R) and which contains a conjugate of , is conjugate to SL(2, R) k 
Q, where O is a closed connected subgroup ofR 2fe . That is, 

H = g (SL(2,R) kO)^ 1 , 

for some go = (Mo;£ ) G G k . Because 

(M ,0)(SL(2,R) tx O)(M ,0)" 1 = SL(2,R) x 

we may take Mo = 1 without loss of generality, and hence 

iT = (l;£ )(SL(2,K)Ktt)(l;-£ ). 
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